The Rraven package is designed to facilitate the exchange of data between R and Raven sound analysis software (Cornell Lab of Ornithology). On one hand, Raven provides very powerful tools for the analysis of (animal) sounds. R, on the other hand, can simplify the automatization of complex routines of analyses. Furthermore, R packages as warbleR, seewave and monitoR (among others) provide additional methods of analysis, working as a perfect complement for those found in Raven. Hence, bridging these applications can largely expand the bioacoustician’s toolkit.
Currently, most analyses in Raven cannot be run in the background from a command terminal. Thus, most Rraven functions are design to simplify the exchange of data between the two programs, and in some cases, export files to Raven for further analysis. This vignette provides detail examples for each function in Rraven, including both the R code as well as the additional steps in Raven required to fully accomplished the analyses. Raven Pro must be installed to be able to run some of the code.
Before getting into the functions, the packages must be installed and loaded. I recommend using the latest developmental version, which is found in github. To do so, you need the R pacakge devtools’ (which of course should be installed!). Some warbleR functions and example data sets will be used, so warbleR should be installed as well:
devtools::install_github("maRce10/warbleR")
devtools::install_github("maRce10/Rraven")
#from CRAN would be
#install.packages("warbleR")
#load packages
library(warbleR)
library(Rraven)
Let’s also use a temporary folder as the working directory i which to save all sound files and data files:
setwd(tempdir())
#load example data
data(list = c("Phae.long1", "Phae.long2", "Phae.long3", "Phae.long4", "selec.table", "selection.files"))
#save sound files in temporary directory
writeWave(Phae.long1,"Phae.long1.wav")
writeWave(Phae.long2,"Phae.long2.wav")
writeWave(Phae.long3,"Phae.long3.wav")
writeWave(Phae.long4,"Phae.long4.wav")
#save Raven selection tables in temporary directory
write.table(selection.files[[1]],file = "100889-Garrulax monileger.selections.txt",
row.names = FALSE, sep= '\t')
write.table(selection.files[[2]],file = "1023-Arremonops rufivirgatus.selections.txt",
row.names = FALSE, sep= '\t')
#this is the temporary directory location (of course different each time is run)
getwd() [1] "/tmp/RtmpnIqxHf"
Importing data from Raven
imp_raven
This function imports Raven selection tables. Multiple files can be imported at once. Raven selection tables including data from multiple recordings can also be imported. It returns a single data frame with the information contained in the selection files. We already have 2 Raven selection tables in the working directory:
list.files(path = tempdir(), pattern = "\\.txt$")[1] "100889-Garrulax monileger.selections.txt" "1023-Arremonops rufivirgatus.selections.txt"
This code shows how to import all the data contained in those files into R:
#providing the name of the column with the sound file names
rav.dat <- imp_raven(all.data = TRUE)
head(rav.dat)| Selection | View | Channel | Begin.Time..s. | End.Time..s. |
|---|---|---|---|---|
| 1 | Spectrogram 1 | 1 | 35.986 | 36.449 |
| 2 | Spectrogram 1 | 1 | 36.461 | 36.906 |
| 3 | Spectrogram 1 | 1 | 37.147 | 37.568 |
| 4 | Spectrogram 1 | 1 | 37.628 | 38.157 |
| 5 | Spectrogram 1 | 1 | 38.151 | 38.530 |
| 6 | Spectrogram 1 | 1 | 38.540 | 38.973 |
| Max.Freq..Hz. | End.File | selec.file | selec.file.1 |
|---|---|---|---|
| 0.0 | 100889-Garrulax monileger.wav | 100889-Garrulax monileger.selections.txt | 100889-Garrulax monileger.selections.txt |
| 0.0 | 100889-Garrulax monileger.wav | 100889-Garrulax monileger.selections.txt | 100889-Garrulax monileger.selections.txt |
| 2411.7 | 100889-Garrulax monileger.wav | 100889-Garrulax monileger.selections.txt | 100889-Garrulax monileger.selections.txt |
| 2454.8 | 100889-Garrulax monileger.wav | 100889-Garrulax monileger.selections.txt | 100889-Garrulax monileger.selections.txt |
| 2454.8 | 100889-Garrulax monileger.wav | 100889-Garrulax monileger.selections.txt | 100889-Garrulax monileger.selections.txt |
| 2368.7 | 100889-Garrulax monileger.wav | 100889-Garrulax monileger.selections.txt | 100889-Garrulax monileger.selections.txt |
Note that the ‘waveform’ view data has been removed. It can also be imported as follows:
rav.dat <- imp_raven(all.data = TRUE, waveform = TRUE)
head(rav.dat)
Raven selections can also be imported in a ‘selection.table’ format so it can be directly input into warbleR functions. To do this you need to set the all.data = FALSE and indicate which column contains the sound file name (using the ‘sound.file.col’ argument):
#providing the name of the column with the sound file names
rav.dat <- imp_raven(sound.file.col = "End.File", all.data = FALSE)
head(rav.dat)| sound.files | channel | selec | start | end | selec.file |
|---|---|---|---|---|---|
| 100889-Garrulax monileger.wav | 1 | 1 | 35.986 | 36.449 | 100889-Garrulax monileger.selections.txt |
| 100889-Garrulax monileger.wav | 1 | 2 | 36.461 | 36.906 | 100889-Garrulax monileger.selections.txt |
| 100889-Garrulax monileger.wav | 1 | 3 | 37.147 | 37.568 | 100889-Garrulax monileger.selections.txt |
| 100889-Garrulax monileger.wav | 1 | 4 | 37.628 | 38.157 | 100889-Garrulax monileger.selections.txt |
| 100889-Garrulax monileger.wav | 1 | 5 | 38.151 | 38.530 | 100889-Garrulax monileger.selections.txt |
| 100889-Garrulax monileger.wav | 1 | 6 | 38.540 | 38.973 | 100889-Garrulax monileger.selections.txt |
The data frame contains the following columns: soundfiles, selec, start, end, and selec.file. You can also import the frequency range parameters in the ‘selection.table’ by setting ‘freq.cols’ tp TRUE (although this columns are not available in our example data). The data frame returned by “imp_raven” (when in the ‘warbleR’ format) can be input into several warbleR functions for further analysis.
extract_ts
The function extracts parameters enconded as time series in Raven selection tables. The resulting data frame can be directly input into functions for time series analysis of acoustic signals as in the warbleR function dfDTW. The function needs an R data frame, so the data should have been previously imported using imp_raven. This example uses the selection_file.ts example data that comes with Rraven:
# freq contour 95
fcts <- extract_ts(X = selection_file_ts, ts.column = "Freq.Contour.95...Hz.")
fcts[,1:14]
fcts[,39:53]| sound.files | selec | F.C.1 | F.C.2 | F.C.3 | F.C.4 | F.C.5 | F.C.6 | F.C.7 | F.C.8 | F.C.9 | F.C.10 | F.C.11 | F.C.12 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Phae.long1.wav | 1 | 7119.1 | 7207.0 | 7382.8 | 7470.7 | 7382.8 | 7470.7 | 7646.5 | 7734.4 | 7734.4 | 7734.4 | 7910.2 | 8173.8 |
| Phae.long1.wav | 2 | 6943.4 | 7119.1 | 7294.9 | 7294.9 | 7470.7 | 7470.7 | 7558.6 | 7646.5 | 7646.5 | 7734.4 | 7646.5 | 8085.9 |
| Phae.long1.wav | 3 | 7031.2 | 7207.0 | 7294.9 | 7294.9 | 7294.9 | 7207.0 | 7207.0 | 7734.4 | 7734.4 | 7910.2 | 8085.9 | 8173.8 |
| F.C.37 | F.C.38 | F.C.39 | F.C.40 | F.C.41 | F.C.42 | F.C.43 | F.C.44 | F.C.45 | F.C.46 | F.C.47 | F.C.48 | F.C.49 | F.C.50 | F.C.51 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 7119.1 | 7207 | 7294.9 | 7294.9 | 7294.9 | 7119.1 | 7119.1 | 7207.0 | 7207.0 | 7294.9 | 7294.9 | 7294.9 | 7294.9 | 7294.9 | 7294.9 |
| 7119.1 | 7207 | 7294.9 | 7382.8 | 7382.8 | 7382.8 | 7294.9 | 7470.7 | 7294.9 | 7382.8 | 7294.9 | 7382.8 | NA | NA | NA |
| 7119.1 | 7207 | 7207.0 | 7382.8 | 7382.8 | 7382.8 | 7382.8 | 7382.8 | 7382.8 | 7382.8 | 7382.8 | 7294.9 | 7294.9 | 7294.9 | 7294.9 |
Note that these sequences are not all of equal length (one has NAs at the end). extract_ts can also interpolate values so all time series have the same length:
# freq contour 95 equal length
fcts <- extract_ts(X = selection_file_ts, ts.column = "Freq.Contour.95...Hz.", equal.length = T)
#look at the last rows wit no NAs
fcts[,21:32]| P.F.19 | P.F.20 | P.F.21 | P.F.22 | P.F.23 | P.F.24 | P.F.25 | P.F.26 | P.F.27 | P.F.28 | P.F.29 | P.F.30 |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 6246.314 | 6373.603 | 6500.893 | 6628.183 | 6755.472 | 6882.762 | 7010.052 | 7137.341 | 7264.631 | 7391.921 | 7519.210 | 7646.5 |
| 6649.379 | 6803.945 | 6958.510 | 7113.076 | 7267.641 | 7422.207 | 7576.772 | 7731.338 | 7885.903 | 8040.469 | 8195.034 | 8349.6 |
| 6834.272 | 7003.993 | 7173.714 | 7343.434 | 7513.155 | 7682.876 | 7852.597 | 8022.317 | 8192.038 | 8361.759 | 8531.479 | 8701.2 |
And the length of the series can also be specified:
# freq contour 95 equal length 10 measurements
fcts <- extract_ts(X = selection_file_ts, ts.column = "Peak.Freq.Contour..Hz.",
equal.length = T, length.out = 10)
knitr::kable(fcts, align = "c", row.names = F)| sound.files | selec | P.F.1 | P.F.2 | P.F.3 | P.F.4 | P.F.5 | P.F.6 | P.F.7 | P.F.8 | P.F.9 | P.F.10 |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Phae.long1.wav | 1 | 3955.1 | 4365.256 | 4775.411 | 5185.567 | 5595.722 | 6005.878 | 6416.033 | 6826.189 | 7236.344 | 7646.5 |
| Phae.long1.wav | 2 | 3867.2 | 4365.244 | 4863.289 | 5361.333 | 5859.378 | 6357.422 | 6855.467 | 7353.511 | 7851.556 | 8349.6 |
| Phae.long1.wav | 3 | 3779.3 | 4326.178 | 4873.056 | 5419.933 | 5966.811 | 6513.689 | 7060.567 | 7607.444 | 8154.322 | 8701.2 |
The time series data frame can be directly input into the dfDTW warbleR function to calculate Dynamic Time Warping distances:
dfDTW(ts.df = fcts) Phae.long1.wav-1 Phae.long1.wav-2 Phae.long1.wav-3
Phae.long1.wav-1 0.000 2665.922 3652.378
Phae.long1.wav-2 2665.922 0.000 2734.500
Phae.long1.wav-3 3652.378 2734.500 0.000relabel_colms
This is a very simple function to relabel columns so they match the selection table format used in warbleR:
#to simplify the example select a subset of the columns
st1 <- selection_file_ts[ ,1:7]
#check original column names
st1| Selection | View | Channel | Begin.Time..s. | End.Time..s. | Low.Freq..Hz. | High.Freq..Hz. |
|---|---|---|---|---|---|---|
| 1 | Spectrogram 1 | 1 | 1.1693549 | 1.3423884 | 2220.1 | 8604.4 |
| 2 | Spectrogram 1 | 1 | 2.1584085 | 2.3214565 | 2169.4 | 8807.1 |
| 3 | Spectrogram 1 | 1 | 0.3433366 | 0.5182553 | 2218.3 | 8756.6 |
# Relabel the basic columns required by warbleR
relabel_colms(st1)| selec | View | Channel | start | end | low.freq | high.freq |
|---|---|---|---|---|---|---|
| 1 | Spectrogram 1 | 1 | 1.1693549 | 1.3423884 | 2220.1 | 8604.4 |
| 2 | Spectrogram 1 | 1 | 2.1584085 | 2.3214565 | 2169.4 | 8807.1 |
| 3 | Spectrogram 1 | 1 | 0.3433366 | 0.5182553 | 2218.3 | 8756.6 |
Additional columns can also be relabeled:
# 2 additional column
relabel_colms(st1, extra.cols.name = c("selec.file", "View"),
extra.cols.new.name = c("Raven selection file", "Raven view"))| selec | Raven view | Channel | start | end | low.freq | high.freq |
|---|---|---|---|---|---|---|
| 1 | Spectrogram 1 | 1 | 1.1693549 | 1.3423884 | 2220.1 | 8604.4 |
| 2 | Spectrogram 1 | 1 | 2.1584085 | 2.3214565 | 2169.4 | 8807.1 |
| 3 | Spectrogram 1 | 1 | 0.3433366 | 0.5182553 | 2218.3 | 8756.6 |
imp_corr_mat
The function imports the output of a batch correlation routine in Raven. Both the correlation and lag matrices contained in the output ‘.txt’ file are read and both waveform and spectrogram (cross-correlation) correlations can be imported.
This example shows how to input the sound files into Raven and how to bring the results back to R. First, the selections need to be cut as single sound files for the Raven correlator to be able to read it. We can do this using the cut_sels function from warbleR:
#create new folder to put cuts
dir.create("cuts")
# add a rowname column to be able to match cuts and selections
selec.table$rownames <- sprintf("%02d",1:nrow(selec.table))
# cut files
cut_sels(X = selec.table, mar = 0.05, path = tempdir(), dest.path = file.path(tempdir(), "cuts"), labels = c("rownames", "sound.files", "selec"), pb = FALSE)
#list cuts
list.files(path = file.path(tempdir(), "cuts"))
Every selection is in it’s own sound file (labeled as `paste(sound.files, selec)). Now open Raven and run the batch correlator on the ‘cuts’ folder as follows:
And then import the output file into R:
# Import output (change the name of the file if you used a different one)
xcorr.rav <- imp_corr_mat(file = "BatchCorrOutput.txt", path = tempdir())
The function returns a list containing the correlation matrix (here only showing the first 5 rows/columns):
xcorr.rav$correlation[1:5, 1:5] 01-Phae.long1-1.wav 10-Phae.long4-2.wav 11-Phae.long4-3.wav 07-Phae.long3-2.wav 05-Phae.long2-2.wav
01-Phae.long1-1.wav 1.000 0.216 0.184 0.285 0.443
10-Phae.long4-2.wav 0.216 1.000 0.781 0.290 0.235
11-Phae.long4-3.wav 0.184 0.781 1.000 0.279 0.186
07-Phae.long3-2.wav 0.285 0.290 0.279 1.000 0.433
05-Phae.long2-2.wav 0.443 0.235 0.186 0.433 1.000
and the time lag matrix:
xcorr.rav$`lag (s)`[1:5, 1:5] 01-Phae.long1-1.wav 10-Phae.long4-2.wav 11-Phae.long4-3.wav 07-Phae.long3-2.wav 05-Phae.long2-2.wav
01-Phae.long1-1.wav 0.000 0.011 0.006 0.028 0.034
10-Phae.long4-2.wav -0.011 0.000 -0.006 0.040 0.023
11-Phae.long4-3.wav -0.006 0.006 0.000 0.046 0.028
07-Phae.long3-2.wav -0.028 -0.040 -0.046 0.000 -0.011
05-Phae.long2-2.wav -0.034 -0.023 -0.028 0.011 0.000
This ouput is ready for stats. For instance, the following code runs a mantel test between cross-correlation (converted to distances) and warbleR spectral parameter pairwise disimilarities:
#convert cross-corr to distance
xcorr.rvn <- 1- xcorr.rav$correlation
#sort matrix to match selection table
xcorr.rvn <- xcorr.rvn[order(rownames(xcorr.rvn)), order(colnames(xcorr.rvn))]
#convert it to distance matrix
xcorr.rvn <- as.dist(xcorr.rvn)
# measure acoustic parameters
sp.wrblR <- specan(selec.table, bp = c(1, 11), wl = 150, pb = FALSE)
#convert them to distance matrix
dist.sp.wrblR <- dist(sp.wrblR)
vegan::mantel(xcorr.rvn, dist.sp.wrblR)
Mantel statistic based on Pearson's product-moment correlation
Call:
vegan::mantel(xdis = xcorr.rvn, ydis = dist.sp.wrblR)
Mantel statistic r: 0.2841
Significance: 0.019
Upper quantiles of permutations (null model):
90% 95% 97.5% 99%
0.160 0.211 0.266 0.310
Permutation: free
Number of permutations: 999
There is actually a good match between two types of analysis!
Exporting R data to Raven
exp_raven
exp_raven saves a selection table in ‘.txt’ format that can be directly opened in Raven. No objects are returned into the R environment. The following code exports a data table from a single sound file:
# Select data for a single sound file
st1 <- selec.table[selec.table$sound.files == "Phae.long1.wav",]
# Export data of a single sound file
exp_raven(st1, file.name = "Phaethornis 1", khz.to.hz = TRUE)
If the path to the sound file is provided the functions exports a ‘sound selection table’, which can be directly open by Raven (and which will also open the associated sound file):
# Select data for a single sound file
st1 <- selec.table[selec.table$sound.files == "Phae.long1.wav",]
# Export data of a single sound file
exp_raven(st1, file.name = "Phaethornis 1", khz.to.hz = TRUE, sound.file.path = tempdir())
This is useful to add new selections or even new measurements:
